The present invention relates to communication methods in general, and to methods for generating multiplexed sequence of media units.
Methods and systems for compressing and transmitting media signals are known in the art. Compressed digital video is largely becoming the preferred medium to transmit video to viewers everywhere. Part of the Moving Pictures Experts Group (MPEG) specifications are standardized methods for compressing and transmitting video. Various audio compression techniques, including MPEG compliant techniques, are also known in the art. In general, MPEG is used today for transmitting video over terrestrial, wireless, satellite and cable communication channels and also for storing digital video.
An audio stream is organized as an ordered sequence of frames. A video stream is usually organized as an ordered sequence of pictures, each picture includes a plurality of pixels. A video picture includes a plurality of slices, each slice including a plurality of macro blocks. The audio and video streams are provided to an audio encoder and video encoder respectively to generate compressed audio and video elementary streams, also referred to as elementary streams.
MPEG compression/encoding utilizes various compression schemes, such as adaptive quantization, intra-frame encoding, inter-frame encoding, run length encoding and variable length coding. Intra-frame coding takes advantage of spatial redundancies in a picture. Inter-frame coding takes advantage of temporal redundancies from picture to picture in a video sequence. Inter-frame coding involves motion estimation and motion compensation. There are three types of motion estimationsxe2x80x94forward, backward and bidirectional. Macroblocks are the elementary unit for motion compensation and adaptive quantization. Each macroblock is associated with a quantization factor field, representative of the degree of quantization. A slice, including a plurality of macroblocks includes a slice header that has a quantization factor field that is associated with some of the macro blocks of the slice.
Elementary streams are packetized to produce PES packets. PES packets made up of elementary streams that form a program share a common time base. The PES packets may also include additional information. PES packets of distinct elementary streams can be arranged as either a Program Stream or a Transport Stream. A Transport Stream combines one or more programs with one or more independent time bases into a single stream. Transport Streams include transport packets of 188 bytes. Transport Stream packets start with a transport packet header. The header includes a packet ID (PID). Transport Stream packets of one PID value carry data of a single elementary stream. Transport Streams include Program Specific Information (PSI) tables. The PSI tables specify which PIDs and accordingly which elementary streams are associated to form each program.
Transport Streams may be of either fixed or variable bit rate. Some programs of the Transport Stream are of a variable bit rate, if, for example, more bits are allocated to complex scenes, and less bits are allocated to more simple scenes.
Transport Streams are provided, usually via a communication medium, to a target decoder, that decodes the encoded elementary streams within the Transport Stream. The decoding of encoded elementary stream is based upon timing information embedded within Transport packets, packs and PES packets. Such timing information includes various time stamps such as decoding time stamp (DTS) and presentation time stamp (PTD), DTS indicating the time that an access unit is decoded at the target decoder to provide a presentation unit, PTS indicating the time that the presentation unit is presented at the target decoder.
The ISO/IEC 13818-1 specification defines a channel as a digital medium that stores or transports a Transport Stream. The aggregate bandwidth of all the components of the Transport Stream must not exceed, at any time, the available bandwidth of the channel. Transport Streams are provided to a channel of a limited available bandwidth/storage space.
Various lossy and lossless techniques are implemented to adapt the aggregate bandwidth of the programs of a Transport Stream to the available bandwidth of a channel. Lossless techniques do not require further compressing of media pictures. Lossless techniques also include delaying or advancing a transmission of elementary data. Lossy techniques involve additional compression, and are usually implemented whenever the appliance of lossless techniques is not feasible or does not provide sufficient results. The further compression usually results in visual quality degradation.
There is a need to provide a method that allows for smoothing data reductions and allocating bit rates in an optimal manner. There is a need for a method that allows for predicting a need for implementing lossy techniques and accordingly spread the data reduction among a relatively large number of access units, programs, elementary streams and/or elementary stream portions.
It is an object of the present invention to provide novel methods for data reduction, and especially for an efficient method for data reduction of data belonging to various programs being transmitted over a single telecommunication channel. The methods involve bandwidth/bit-rate limitations predictions, and allowing for changing the timing of a transmission of media units. A media unit is a set, a sequence or a group of signals representative of audio and/or visual content. Media unit can include at least one macroblock, but can also include portions of macroblocks. It is noted that if the data reduction includes motion compensation and/or adaptive quantization than the media unit includes at least one macroblock.
The invention provides a method that includes calculating a sequence time threshold, for each received media units sequence, a sequence usually includes groups of macroblocks, and executing modification and replacement of media units in view of the sequence time threshold.
The invention provides a method for generating a multiplexed sequence, the method includes the steps of: (a) receiving at least one media unit sequence; (b) calculating a sequence time threshold, for each received media unit sequence, in response to: (i) timing information embedded within the media unit sequence, (ii) an estimated allocated bit rate of the media unit sequence, and (iii) aggregate size of at least some media units belonging to the media unit sequence; (c) determining, for each media unit sequence, which media units to append to the multiplexed sequence and which media units to modify to provide corresponding modified media units, in response to the sequence time threshold; and (d) modifying media units, appending media units and corresponding modified media units, in response to the determination.
The invention provides a method for generating a multiplexed sequence, the method includes the steps of: (I) receiving at least one media unit sequence; (II) defining, for each media unit sequence, a media unit sub-sequence comprising media units that are complying with a sub-sequence timing constraint; (III) for each media unit sub-sequence performing the steps of: (III.I) calculating a sub-sequence time threshold, in response to: (a) timing information embedded within the media unit sub-sequence, (b) an estimated allocated bit rate of the media unit sub-sequence, and (c) aggregate size of at least some media units belonging to the media unit sub-sequence; (III.II) decreasing the size of the media unit sub-sequence, by modifying at least one media unit belonging to the media unit sub-sequence to generate at least one corresponding modified media unit, in response to a bit rate allocated to the media unit sub-sequence; and (III.III) appending the at least one corresponding modified media unit in place of the at least one media unit to the multiplexed sequence, if the at least one corresponding modified media unit was generated during the step of decreasing the size of the media unit sub-sequence.
The invention provides a method for generating a multiplexed sequence, the method includes the steps of: (aa) receiving at least one media unit sequence; (bb) for each media unit sequence performing the steps of: (bb.1) calculating a sequence time threshold, in response to: (a) timing information embedded within the media unit sequence, (b) an estimated allocated bit rate of the media unit sequence, and (c) aggregate size of at least some media units belonging to the media unit sequence; (bb2) determining a target size of a media unit group in response to the corresponding sequence time threshold, the media unit group comprising at least some of the media units of the sequence; (bb3) decreasing the size of the media unit group by modifying at least one media unit of the media unit group, to generate at least one corresponding modified media unit, if the size of the media unit group is greater than the target size; and (bb4) appending at least one corresponding modified media unit in place of the media units to the multiplexed sequence, if the at least one corresponding modified media unit was generated during the step of decreasing the size of the media unit group.
The invention provides a method in which an allocated estimated bit rate can be responsive to a media unit sequence priority. Data reduction of media units can also be responsive to a media unit sequence priority.